System and method of augmenting images of a user

ABSTRACT

A method to be performed by at least one computer processor forming at least a part of a computing system comprising a mobile computing device comprising a camera, the method including obtaining a set of images of a head of a user by moving the mobile computing device through one or more two- and three-dimensional paths about the head; identifying, from the set of images, landmarks of the head of the user, wherein the landmarks comprise ears, nose, eyes, eyebrows, chin, cheeks, neckline, and hairline of the user; generating, based on the identified landmarks, a user framework; and scaling and adjusting, based on the user framework, a set of hairstyles from a hairstyle database based on hair dimensions, characteristics, and color with the landmarks, the hairstyle database being synchronized with a user profile and based one or more hairstyle selections by the user.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/951,876, filed Nov. 18, 2020, which is a continuation of U.S. patentapplication Ser. No. 15/810,055, filed Nov. 11, 2017, which claimspriority to U.S. provisional patent application No. 62/421,052, entitled“Augmenting facial images” and filed Nov. 11, 2016. The contents of theUnited States provisional patent application is incorporated herein byreference in its entirety as if set forth verbatim.

FIELD OF THE INVENTION

The subject matter described herein relates to modifying image data andin particular modify image data containing an image of a face.

BACKGROUND OF THE INVENTION

Determining hairstyles that suit a person can be challenging. A customerof a salon may not know the hairstyle they would like because they donot know whether the hairstyle will suit their face. The solution ofthis disclosure resolves these and others of the art.

BRIEF SUMMARY OF THE INVENTION

In one aspect, a method is provided. The method can include obtaining aset of images of a body part of a user, such as the head. Landmarks forthe body party of the user can be identified in the images. A userframework can be generated based on the identified landmarks. A set ofaesthetic expressions (e.g., hairstyles, tattoos, articles of clothing,etc.) can be selected, customized, adjusted, and/or scaled based on theuser framework. In those embodiments where the hairstyle is theaesthetic expression, the selecting, customizing, adjusting, and/orscaling of the set of hairstyles conforming the set of hairstyles to thesize and shape of the head of the user in the set of images. An image ofthe user's body part can be displayed on a user device. The image of theuser's body part can be augmented with at least one set of thehairstyles.

In some embodiments, a system and method is disclosed to be performed byat least one computer processor forming at least a part of a computingsystem. The method can include obtaining a set of images of a body partof a user; identifying, from the set of images, landmarks of the bodypart of the user; generating, based on the identified landmarks, a userframework; adjusting, based on the user framework, a set of aestheticexpressions; and displaying, on a display device, an image of the user'sbody part augmented with at least one set of aesthetic expressions.

In some embodiments, the body part is a head and the aestheticexpressions are hairstyles.

In some embodiments, the aesthetic expressions are accessible in adatabase that is synchronized with a publicly and/or privately availabledatabase of aesthetic expressions according to one or more useridentified criteria. The criteria can include aesthetic expressions ofcelebrities, social media, criteria of contacts within a network,network suggested criteria based on user information (e.g., age,ethnicity, nationality, gender, user or system identified trends), andthe like.

In some embodiments, the body part is a location on the user forreceiving a tattoo and the aesthetic expressions are tattoos.

In some embodiments, the computing system is a mobile computing devicecomprising a camera, wherein the set of images are obtained by movingthe mobile computing device through one or more two- and/orthree-dimensional paths about the body part.

In some embodiments, the method includes presenting an instruction onthe display device that causes the user to maintain the mobile computingdevice at one location relative to a predetermined location of the bodypart; and positioning the camera of the mobile computing device towardsthe predetermined location to obtain the set of images.

In some embodiments, the method includes communicating an indication, bythe mobile communications device, to the user regarding an outputassociated with analyzing the set of images and/or the aestheticexpression.

In some embodiments, the method includes maintaining the mobilecomputing device at one location relative to a predetermined location ofthe body part; positioning the camera of the mobile computing devicetowards the predetermined location to obtain the set of images; andmoving the body part across a predetermined path or across a pluralityof different body part orientations. The step of moving can includestarting the moving of the mobile device behind a lateral midline of thebody part; and terminating the moving of the mobile device at a locationdifferent than the lateral midline of the body part. The step of movingcan include positioning the mobile device along a plurality ofpredetermined positions along a predetermined three-dimensional path. Insome embodiments, the plurality of positions are determinedautomatically or manually by analyzing the set of images and identifyingone or more landmarks of the body part of the user.

In some embodiments, the method includes stitching together the set ofimages; and forming the user framework with a three-dimensional map ofthe user's body part. The three-dimensional map can include thelandmarks detected by analyzing the set of images that are associatedwith the body part. The map can include a multi-color graphic withindicators associated with the one or more landmarks

In some embodiments, the body part is the head and the one or morelandmarks include one or more features of the head.

In some embodiments, a method and system is disclosed to be performed byat least one computer processor forming at least a part of a computingsystem. The method includes obtaining a set of images of a body part ofa user; identifying, from the set of images, landmarks of the body partof the user; generating, based on the identified landmarks, a userframework; adjusting, based on the user framework, a set of aestheticexpressions; selecting, by the user, an aesthetic expression from theset of aesthetic expressions; displaying, on a display device, an imageof the user's body party augmented with the selected aestheticexpression.

In some embodiments, the computing system is a mobile computing devicehaving the display device, wherein selecting, by the user, iscommunicated in a graphical user interface of the display device.

In some embodiments, steps of the method are stored in a set ofinstructions in a non-transitory computer readable storage medium of thecomputing system.

In some embodiments, the body part is a head and the aestheticexpressions are hairstyles, and wherein the hairstyles comprising athree-dimensional framework that is adjusted to the user framework.

Implementations of the current subject matter can include, but are notlimited to, methods consistent with the descriptions provided herein aswell as articles that comprise a tangibly embodied machine-readablemedium operable to cause one or more machines (e.g., computers, etc.) toresult in operations implementing one or more of the described features.Similarly, computer systems are also described that may include one ormore processors and one or more memories coupled to the one or moreprocessors. A memory, which can include a computer-readable storagemedium, may include, encode, store, or the like one or more programsthat cause one or more processors to perform one or more of theoperations described herein. Computer implemented methods consistentwith one or more implementations of the current subject matter can beimplemented by one or more data processors residing in a singlecomputing system or multiple computing systems. Such multiple computingsystems can be connected and can exchange data and/or commands or otherinstructions or the like via one or more connections, including but notlimited to a connection over a network (e.g. the Internet, a wirelesswide area network, a local area network, a wide area network, a wirednetwork, or the like), via a direct connection between one or more ofthe multiple computing systems, etc.

The details of one or more variations of the subject matter describedherein are set forth in the accompanying drawings and the descriptionbelow. Other features and advantages of the subject matter describedherein will be apparent from the description and drawings, and from theclaims. While certain features of the currently disclosed subject matterare described for illustrative purposes in relation to a smartphoneapplication, it should be readily understood that such features are notintended to be limiting. The claims that follow this disclosure areintended to define the scope of the protected subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, show certain aspects of the subject matterdisclosed herein and, together with the description, help explain someof the principles associated with the disclosed implementations. In thedrawings,

FIG. 1 is an illustration of a system having one or more featuresconsistent with the present description;

FIG. 2 is an illustration of using the mobile device to capture an imageof the user 204 associated with the mobile device;

FIG. 3 is an illustration of using the mobile computing device tocapture an image of the user associated with the mobile computingdevice;

FIG. 4 illustrates a graphical user interface of a system having one ormore features consistent with the present description;

FIG. 5 is an illustration of a graphical user interface having one ormore features consistent with the present description;

FIG. 6 is an illustration of a graphical user interface having one ormore features consistent with the present description;

FIG. 7 is an illustration of a graphical user interface have one or morefeatures consistent with the present description;

FIG. 8 is a schematic illustration of elements of an application for usewith a system having one or more features consistent with the presentdescription; and

FIG. 9 illustrates a method 900 having one or more features consistentwith the present description.

DETAILED DESCRIPTION OF THE INVENTION

The ubiquity of smartphone technology has enabled the automation and/orthe simplification of many aspects of our day-to-day lives. One of theexceptions is the ability to augment an image of a user of a mobiledevice to determine a preferred style.

The presently described subject matter is directed to obtaining one ormore images of a user of a mobile device, using a camera of the mobiledevice, to facilitate augmenting that image with different preselectedstyles to determine a preferred style by the user.

In this application, the term “aesthetic expression” can mean ahairstyle, makeup, a tattoo, an article of clothing on a user, jewelry,or the like.

FIG. 1 is an illustration of a system 100 having one or more featuresconsistent with the present description. The system 100 can include aserver 102. The server 102 can include a memory 104. The memory 104 caninclude a database. The memory 104 can include one or more styles withwhich to augment an image(s) of a user of a mobile device. The system100 can include a mobile computing device 106. The mobile computingdevice 106 can be associated with a user. The mobile computing device106 can include memory. The memory can be configured to storecomputer-readable instructions forming an application for execution onthe mobile computing device 106. The mobile computing device 106 can beconfigured to facilitate analysis of a body part such as the head andselection of an aesthetic expression (e.g., a hairstyle) for the user ofthe mobile computing device 106. It is to be understood that any of theherein disclosed embodiments could also be applied to other areas forscaling, adjusting, customizing, or otherwise selecting appearance of auser, including tattoos on a body part, one or more clothes on a user,or the like. For example, any of the herein disclosed system and methodscould be configured so an end-user can scan a body part and then selecta tattoo for placement on the body part.

In certain embodiments, to facilitate selection of an aestheticexpression such as a hairstyle for the user, the mobile computing device106 can be configured to obtain one or more images of the user's head.The mobile computing device 106 can be configured to facilitateaugmenting of the image of the user's head with various hairstyles.

The mobile computing device 106 can include a camera. The mobilecomputing device 106 can be configured to use the camera to obtain animage of the user's head. The mobile computing device 106 can beconfigured to obtain a panoramic image of the user's head. Instructionscan be provided to the user by the application through a display of themobile computing device 106. The mobile computing device 106 can beconfigured to obtain images of the user's head in a number of differentways. The application can be configured to present instructions to theuser consistent with the different ways of obtaining images of theuser's head.

FIG. 2 is an illustration of using the mobile device 202 to capture animage of the user 204 associated with the mobile device 202. In theexample illustrated in FIG. 2, the mobile computing device 202 the usercan move the mobile computing device 202 through one or more two- and/orthree-dimensional paths 206 about the user's head. In certainembodiments, the images obtained by the mobile computing device 202 caninclude a plurality of images that are capable of being stitchedtogether to form a three-dimensional map of the user's head. When themobile computing device 202 is moved through the path 206, the camera208 can be directed toward and/or about the user 204. In somevariations, the path 206 may start and end behind the lateral midline210 of the head of the user 204. In other variations, the path 206 maystart and end at some other point, for example, at the lateral midline210 of the head of the user 204.

The mobile computing device 202 can be configured to obtain a pluralityof images as it is moved along the path 206. The mobile computing device202 can be configured to take images at set points along the path 206(e.g., a combination of one or more of x, y, and z coordinates). Forexample, the application running on the mobile computing device 202 canbe configured to determine that the mobile computing device 202 is at aparticular location along the path 206 and cause the camera 208 toobtain an image of the head of the user 204 at that location. Theparticular location(s) in certain embodiments can be determinedautomatically or manually by one or more landmarks of the head of theuser. Examples of landmarks can include the user's ears, eyes, nose,mouth, chin, forehead, cheeks, eye brows, eye lashes, and/or the like.The mobile computing device 202 can be configured to use the landmarksas base locations in a predetermined set order to determine where themobile computing device 202 is along the path 206. For example, in oneembodiment, the path 206 may consist moving from the left ear of theuser 204, then to the nose and mouth, and then to the right ear of theuser.

FIG. 3 is an illustration of using the mobile computing device 302 tocapture an image of the user 304 associated with the mobile computingdevice 302. In this example, the user is instructed to keep the mobilecomputing device 302 at one location with the camera 306 of the mobilecomputing device 302 pointing toward the user 304. The application cancause an instruction to be presented to the user on a display of themobile computing device 302. The instruction can instruct the user tomove their body part(s) (e.g., rotate their head) with the camera 306pointed toward them.

FIG. 4 illustrates a graphical user interface of a system having one ormore features consistent with the present description. In somevariations, the mobile computing device 106, as illustrated in FIG. 1,can be configured to request that the user position the mobile computingdevice 106 at a particular location relative to the user's body part. Instep (a), the user is prompted by the display of device 106 to move hisbody part (e.g., head) to one or more positions or along a predeterminedpath. In step (b), the user has begun moving his head along the one ormore positions or along the predetermined path. Once positioned inaccordance with the instructions, at step (c), an indication will becommunicated to the user via the display of device 106. It is understoodthat the instructions of FIG. 4 can be only one position by the user orcan be the user moving between a plurality of positions or movingthrough a pre-determined path. Whether only one or the plurality ofpositions and/or the pre-determined path, the user can be notified oncethe device 106 has captured the information so as to communicate itsoutput in step (c). While a male individual is depicted in FIG. 4, anyperson or mammal, including any gender, is also contemplated for use inthis particular embodiment.

Referring back to FIG. 1. In some variations, the mobile computingdevice 106 can be configured to use the obtained images of the user'sbody part to generate a user framework.

In other variations, the mobile computing device 106 can be configuredto transmit the images of the user's body part to a server 102. Themobile computing device 106 can be in electronic communication with aserver 102 through a network 108, such as the Internet. The server 102can be configured to generate a two- and/or three-dimensional userframework. A user framework can be a two- and/or three-dimensionalframework of a user's head that include an output map that depicts theuser's body part (e.g., the user's head) with one or more detectedlandmarks. The output may also be a multi-color graphic with indicatorsassociated with characteristics of the one or more detected landmarks.To generate the framework of the user's head, the images of the user'shead can be used to determine the dimensions of the user's head,including dimensions and spatial arrangement of one or more detectedlandmarks. The framework can include an indication of the relativepositions and sizes of various elements of the user's head, includingone or more detected landmarks. For example, the user framework caninclude an indication of the relative position, size and/or shape oflandmarks such as the user's ears, nose, eyes, neck, skull, cheeks,eyebrows, hairline, or the like.

The user framework can be used by the mobile computing device 106 and/orthe server 102 to scale, adjust, or otherwise customize aestheticexpressions (e.g., hairstyles) of an aesthetic expression repository(e.g., a database of hairstyles, tattoos, clothes, jewelry etc.).Aesthetic expressions can be stored in one or more of the mobilecomputing device 106, the server 102, memory 104, or the like.

In those embodiments, where aesthetic expressions are hairstyles, eachhairstyle stored in the hairstyle repository may have a hairstyleframework. The hairstyle framework can include an indication of therelative positions of various elements of the hairstyle framework.Certain elements of the hairstyle framework may be married with therelative positions of a user framework. Certain elements of thehairstyle framework may be customized by or for the user, including hairdimensions, characteristics, and/or color with the relative positions ofa user framework. By scaling, adjusting, and/or customizing thehairstyle framework with the user framework, individual hairstyles canbe scaled, adjusted, and/or customized to individual users.

Following scaling, adjustment, and/or customization, the hairstyles inthe hairstyle repository can be scaled, adjusted, customized, and/orpresented to the user. FIG. 5 is an illustration of a graphical userinterface 500 having one or more features consistent with the presentdescription. The graphical user interface 500 can be configured topresent hairstyles 502 on an image 504 of a user's head. The hairstyles502 presented to the user on the graphical user interface 500 can bescaled to the user's head, including one or more landmarks detectedthereon, thereby appearing as having the proper size and shape. If themobile computing device 506 has a touchscreen display, a user can modifyor customize a presented hairstyle (e.g., modify one or morecharacteristics of the presented hairstyle), and/or scroll throughvarious hairstyles by swiping on the screen of the touchscreen display,interacting with an on screen element 508, or the like.

The mobile computing device 506 can be configured to facilitate rotatingof the two- and/or three-dimensional image 504 of the head of the user.The mobile computing device 506 can include a touchscreen. In responseto an input from a user, through the touchscreen, the image 504 of headof the user can be rotated to see the sides, back, or the like of theimage 504 of the head of the user.

In some variations, a user may select a subset of hairstyles priordisplaying them on an image of the user's head. In response to selectionof a subset of hairstyles of a hairstyle repository, only the subset ofhairstyles may be scaled using the user framework. In some embodiments,instead of hairstyles the subset can include clothes or tattoos or thelike to facilitate a user's selection.

In some variations, the application can be configured to facilitateselection of a hairstyle from a hairstyle gallery. FIG. 6 is anillustration of a graphical user interface having one or more featuresconsistent with the present description. FIG. 6 illustrates a graphicaluser interface where a user can choose to select a subset of varioushairstyles. Selections can include, for example, facial hairstyles, malehairstyles, female hairstyles or the like. Selections can also include,the length of the hair, the style of the hair, the color of the hair,the type of the hair, or the like. In some variations, a user can sorttheir hairstyle lists by various celebrities, or search for a particularcelebrity to select their hairstyle.

FIG. 7 is an illustration of a graphical user interface 700 have one ormore features consistent with the present description. The graphicaluser interface 700 can include a user profile. A user of the presentapplication can save hairstyles to their profile. The user profile caninclude information about a user's stylist, products used by the user,products used by the stylist for the user, or the like.

When a user selects hairstyles, a user profile may be updated on aserver 102. A companion application may be available to a stylist. Forexample, a stylist may be using another mobile computing device 110. Thestylist may be able to access the user's saved hairstyles and reviewthem with the user. When a stylist uses products on a user during anappointment, the stylist may provide an indication through the companionapplication that they have applied that product, or used a particulartool when styling the user's hair. This can then appear in the user'sprofile.

The application may be configured to facilitate purchase of the productby the user. A revenue sharing scheme can be implemented whereby thestylist receives a share of the revenue generated through the user'spurchase of the products through the application.

FIG. 8 is a schematic illustration of elements of an exemplaryapplication for use with an exemplary system having one or more featuresconsistent with the present description. It is to be understood that theschematic illustration of FIG. 8 can be adapted for other aestheticexpression applications outside of hair, including tattoo(s) on a user,clothes on a user, and the like. In some variations, a user can use theapplication to book an appointment with a stylist, a tattoo artist, orthe like. The user can book an appointment with their own stylist orfind a new stylist or a tattoo artist for a new tattoo with which tobook an appointment. Users can sort stylist by location, rating,availability or the like. A companion application for the stylist can beprovided that allows the stylist to manage their schedule. The companionapplication can facilitate communication between the stylist and theuser.

The application can facilitate purchase of haircare products. Stylistscan recommend haircare products based on a hairstyle they have given tothe client and/or the products that they used on the user. Similarly,when a user selects a particular hairstyle that they would like to have,products that are typically used to create that hairstyle can berecommended to the user.

A user profile, such as the user profile described with respect to FIG.7 can include a headshot of the user with the hairstyle that theyselected. The user can upload the image to social media. The applicationcan facilitate crowdsourcing of hairstyles that may be suitable for aparticular user.

In some variations, the application can facilitate the stylist taking aphotograph of the user after the user's hair has been styled. Thephotograph of the user can be stored in a database and associated withthe user.

The application can facilitate the determination of various features ofa hairstyle associated with a user and store those features of thehairstyle. For example, after a stylist has taken an image of thehairstyle of the user, the application can be configured to determinethe length of the sides of the hair, the length of the back of the hair,the hair-type of the user, the style type selected, or the like. Theapplication can take note of the stylist that styled the hair. Whenstyling the user's hair, the stylist can take a note of the productsused when styling the hair. The stylist can provide style tips for theuser to follow when they get home.

In some variations, video data can be obtained while the user is havingtheir hair styled and the application can be configured to facilitateplayback of the video for the user to follow when applying productand/or styling their own hair.

In other embodiments, the application can facilitate the determinationof various features of a tattoo associated with a user and store thosefeatures of the tattoo. For example, after a tattoo artist or the userhas taken an image of a tattoo and/or a body part of the user, theapplication can be configured to determine dimensions and other stylizedattributes of the tattoo, corresponding style type, or the like. Theapplication can take note of the artist that drew and/or implemented thetattoo on the user.

The server 102 can include a processor (112). Processor(s) 112 can beconfigured to provide information processing capabilities to a computingdevice having one or more features consistent with the current subjectmatter. The computing device can be, for example, a smart phone 106,tablet 110, desktop 114, or the like. Processor 112 may include one ormore of a digital processor, an analog processor, a digital circuitdesigned to process information, an analog circuit designed to processinformation, a state machine, and/or other mechanisms for electronicallyprocessing information. Although processor 112 is shown in FIG. 1 as asingle entity, this is for illustrative purposes only. In someimplementations, processor 112 may include a plurality of processingunits. These processing units may be physically located within the samedevice, or processor 112 may represent processing functionality of aplurality of devices operating in coordination. For example, some of thefunctions of processor 112 may be performed by the mobile computingdevice 106, the server 102, other computing devices 110, 114, or thelike. The processor 112 may be configured to execute machine-readableinstructions, which, when executed by the processor 112 may cause theprocessor 112 to perform one or more of the functions described in thepresent description. The functions described herein may be executed bysoftware; hardware; firmware; some combination of software, hardware,and/or firmware; and/or other mechanisms for configuring processingcapabilities on processor 112.

Electronic storage 104 may comprise electronic storage media thatelectronically stores information. The electronic storage media ofelectronic storage 104 may include one or both of system storage that isprovided integrally (i.e., substantially non-removable) with a computing64046 device, such as a smartphone, tablet, smartwatch, desktop, server,or the like, and/or removable storage that is removably connectable toserver xx via, for example, a port (e.g., a USB port, a firewire port,etc.) or a drive (e.g., a disk drive, etc.). Electronic storage 104 mayinclude one or more of optically readable storage media (e.g., opticaldisks, etc.), magnetically readable storage media (e.g., magnetic tape,magnetic hard drive, floppy drive, etc.), electrical charge-basedstorage media (e.g., EEPROM, RAM, etc.), solid-state storage media(e.g., flash drive, etc.), and/or other electronically readable storagemedia. The electronic storage xx may include one or more virtual storageresources (e.g., cloud storage, a virtual private network, and/or othervirtual storage resources). Electronic storage 104 may store softwarealgorithms, information determined by processor 112, informationreceived from one or more computing devices, such as server 102, clientcomputing devices, such as mobile computing device 106, information thatenables the one or more computing device to function, or the like.

In some variations, external resources 116 can provide additionalinformation or add-on information. For example, the application executedon the mobile computing device 106 and/or supported by the server 102can be configured to obtain product information from productmanufacturers, hairstyle information from other stylists or celebritystylists, or the like. In other embodiments, this information can relateto tattoos of a particular celebrity, tattoos by a particular artist(including well-known tattoo artists), an article of clothing for acelebrity (e.g., a dress worn by an actress at an event), an article ofclothing by a particular company or fashion designer, an article ofclothing suggested by a publication, jewelry worn by a celebrity,jewelry by a designer, or the like. This information can be obtained bythe mobile computing device 106 and/or the server 102 from one or moreexternal resources 116.

FIG. 9 illustrates a method 900 having one or more features consistentwith the current subject matter. The operations of method 900 presentedbelow are intended to be illustrative. In some embodiments, method 900may be accomplished with one or more additional operations notdescribed, and/or without one or more of the operations discussed.Additionally, the order in which the operations of method 900 areillustrated in FIG. 9 and described below is not intended to belimiting.

In some embodiments, method 900 may be implemented in one or moreprocessing devices (e.g., a digital processor, an analog processor, adigital circuit designed to process information, an analog circuitdesigned to process information, a state machine, and/or othermechanisms for electronically processing information). The one or moreprocessing devices may include one or more devices executing some or allof the operations of method 900 in response to instructions storedelectronically on an electronic storage medium. The one or moreprocessing devices may include one or more devices configured throughhardware, firmware, and/or software to be specifically designed forexecution of one or more of the operations of method 900.

At 902, a set of images of a body part, such as a head, can be obtained.The images can be obtained using one or more methods described herein,especially with reference to FIG. 2 and FIG. 3. In one variation, a usercan use a camera of a mobile computing device, such as mobile computingdevice 106, to obtain the set of images of the body part of a user.

At 904, landmarks of the body part, such as the head, can be identifiedfrom the set of images of the head of the user. Landmarks of the headcan include the user's ears, nose, eyes, eyebrows, chin, cheeks,neckline, hairline, or the like. The landmarks can be identified usingimage analysis. In other embodiments where the body part is somethingother than a head, the landmarks are defined according to the particularbody part (e.g., where the body part is a hand, the landmark can includeone or more of the fingers, palm, etc.). The identifying of thelandmarks can be performed by the server 102, mobile computing device106, or the like.

At 906, a user framework can be generated based on the identifiedlandmarks. The user framework can include an indication of the relativesize and position of the identified landmarks.

At 908, a set of aesthetic expressions (e.g., hairstyles) can beselected, adjusted, scaled, and/or otherwise customized based on theuser framework. The hairstyles can be scaled so that the shape and sizeof the hairstyle can be matched to the shape and size and positions ofthe landmarks of the user's head. This can allow the set of hairstylesto be presented to the user on a display device in a manner where thehairstyles look natural on the user's head.

At 910, the image of the user's body party can be augmented with atleast one of the set of hairstyles. The augmented image can be displayedon display device, such as a display of a mobile computing device. It isto be understood that the method illustrated in FIG. 9 is merelyexemplary and other steps could be included. Method 900 can also beadapted for uses outside of the hairstyle space, including with otheraesthetic expressions such as tattoos, clothes, jewelry not only aboutthe head but with other body part(s) of the user.

Without in any way limiting the scope, interpretation, or application ofthe claims appearing herein, a technical effect of one or more of theexample embodiments disclosed herein may include facilitating augmentingimages of a user's head with different hairstyles.

One or more aspects or features of the subject matter described hereincan be realized in digital electronic circuitry, integrated circuitry,specially designed application specific integrated circuits (ASICs),field programmable gate arrays (FPGAs) computer hardware, firmware,software, and/or combinations thereof. These various aspects or featurescan include implementation in one or more computer programs that areexecutable and/or interpretable on a programmable system including atleast one programmable processor, which can be special or generalpurpose, coupled to receive data and instructions from, and to transmitdata and instructions to, a storage system, at least one input device,and at least one output device. The programmable system or computingsystem may include clients and servers. A client and server aregenerally remote from each other and typically interact through acommunication network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

These computer programs, which can also be referred to programs,software, software applications, applications, components, or code,include machine instructions for a programmable processor, and can beimplemented in a high-level procedural language, an object-orientedprogramming language, a functional programming language, a logicalprogramming language, and/or in assembly/machine language. As usedherein, the term “machine-readable medium” refers to any computerprogram product, apparatus and/or device, such as for example magneticdiscs, optical disks, memory, and Programmable Logic Devices (PLDs),used to provide machine instructions and/or data to a programmableprocessor, including a machine-readable medium that receives machineinstructions as a machine-readable signal. The term “machine-readablesignal” refers to any signal used to provide machine instructions and/ordata to a programmable processor. The machine-readable medium can storesuch machine instructions non-transitorily, such as for example as woulda non-transient solid-state memory or a magnetic hard drive or anyequivalent storage medium. The machine-readable medium can alternativelyor additionally store such machine instructions in a transient manner,such as for example as would a processor cache or other random accessmemory associated with one or more physical processor cores.

To provide for interaction with a user, one or more aspects or featuresof the subject matter described herein can be implemented on a computerhaving a display device, such as for example a cathode ray tube (CRT) ora liquid crystal display (LCD) or a light emitting diode (LED) monitorfor displaying information to the user and a keyboard and a pointingdevice, such as for example a mouse or a trackball, by which the usermay provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well. For example, feedbackprovided to the user can be any form of sensory feedback, such as forexample visual feedback, auditory feedback, or tactile feedback; andinput from the user may be received in any form, including, but notlimited to, acoustic, speech, or tactile input. Other possible inputdevices include, but are not limited to, touch screens or othertouch-sensitive devices such as single or multi-point resistive orcapacitive trackpads, voice recognition hardware and software, opticalscanners, optical pointers, digital image capture devices and associatedinterpretation software, and the like.

In the descriptions above and in the claims, phrases such as “at leastone of or “one or more of may occur followed by a conjunctive list ofelements or features. The term “and/or” may also occur in a list of twoor more elements or features. Unless otherwise implicitly or explicitlycontradicted by the context in which it used, such a phrase is intendedto mean any of the listed elements or features individually or any ofthe recited elements or features in combination with any of the otherrecited elements or features. For example, the phrases “at least one ofA and B;” “one or more of A and B;” and “A and/or B” are each intendedto mean “A alone, B alone, or A and B together.” A similarinterpretation is also intended for lists including three or more items.For example, the phrases “at least one of A, B, and C;” “one or more ofA, B, and C;” and “A, B, and/or C” are each intended to mean “A alone, Balone, C alone, A and B together, A and C together, B and C together, orA and B and C together.” Use of the term “based on,” above and in theclaims is intended to mean, “based at least in part on,” such that anunrecited feature or element is also permissible. The subject matterdescribed herein can be embodied in systems, apparatus, methods, and/orarticles depending on the desired configuration. The implementations setforth in the foregoing description do not represent all implementationsconsistent with the subject matter described herein. Instead, they aremerely some examples consistent with aspects related to the describedsubject matter. Although a few variations have been described in detailabove, other modifications or additions are possible. In particular,further features and/or variations can be provided in addition to thoseset forth herein. For example, the implementations described above canbe directed to various combinations and subcombinations of the disclosedfeatures and/or combinations and subcombinations of several furtherfeatures disclosed above. In addition, the logic flows depicted in theaccompanying figures and/or described herein do not necessarily requirethe particular order shown, or sequential order, to achieve desirableresults. Other implementations may be within the scope of the followingclaims.

What is claimed is:
 1. A method to be performed by at least one computerprocessor forming at least a part of a computing system comprising amobile computing device comprising a camera, the method comprising:obtaining a set of images of a head of a user by moving the mobilecomputing device through one or more two- and three-dimensional pathsabout the head; identifying, from the set of images, landmarks of thehead of the user, wherein the landmarks comprise at least two of ears,nose, eyes, eyebrows, chin, cheeks, neckline, and hairline of the user;generating, based on the identified landmarks, a user framework;scaling, based on the user framework, a set of hairstyles from ahairstyle database based on hair dimensions, characteristics, and colorwith the landmarks, the hairstyle database being synchronized with auser profile and based one or more hairstyle selections by the user, thehairstyle database comprising hairstyles of celebrities sourced fromsocial media; displaying, on a display device, an image of the user'shead augmented with at least one set of hairstyles; synchronizing a userprofile based on the user with a publicly and privately availabledatabase of hairstyles; saving, by the user through the mobile computingdevice, hairstyles to the user profile; and accessing, by a hair stylistthrough a second mobile computing device, the user profile.
 2. Themethod of claim 1, further comprising synchronizing, by the stylist,hair products, hair tools, and hairstyle selections of the user profilefrom hair appointments by the user.
 3. The method of claim 1recommending, by the database, one or more hair products to the userbased on the user profile and hairstyle selections by the user.
 4. Themethod of claim 1, further comprising: moving the mobile computingdevice through one or more two- and three-dimensional paths about thehead; presenting an instruction on the display device that causes theuser to temporarily maintain the mobile computing device at one locationrelative to a predetermined location of the head; and temporarilypositioning the camera of the mobile computing device towards thepredetermined location to obtain the set of images.
 5. The method ofclaim 1, further comprising: communicating an indication, by the mobilecommunications device, to the user regarding an output associated withanalyzing the set of images and/or the hairstyle.
 6. The method of claim4, wherein the step of moving comprises: starting the moving of themobile device behind a lateral midline of the head; and terminating themoving of the mobile device at a location different than the lateralmidline of the head.
 7. The method of claim 4, wherein the step ofmoving comprises: positioning the mobile device along a plurality ofpredetermined positions along a predetermined three-dimensional path. 8.The method of claim 7, wherein the plurality of positions are determinedautomatically by analyzing the set of images and identifying one or morelandmarks of the head of the user.
 9. The method of claim 1, furthercomprising: stitching together the set of images; and forming the userframework with a three-dimensional map of the user's head.
 10. Themethod of claim 9, wherein the three-dimensional map comprises thelandmarks detected by analyzing the set of images that are associatedwith the head.
 11. The method of claim 10, wherein the three-dimensionalmap comprises a multicolor graphic with indicators associated with theone or more landmarks.
 12. A method to be performed by at least onecomputer processor forming at least a part of a computing systemcomprising a mobile computing device comprising a camera, the methodcomprising: obtaining a set of images of a head of a user by moving themobile computing device through one or more two- and three-dimensionalpaths about the head; identifying, from the set of images, landmarks ofthe head of the user, wherein the landmarks comprise at least two ofears, nose, eyes, eyebrows, chin, cheeks, neckline, and hairline of theuser; generating, based on the identified landmarks, a user framework;scaling, based on the user framework, a set of hairstyles from ahairstyle database based on hair dimensions, characteristics, and colorwith the landmarks, the hairstyle database being synchronized with auser profile and based one or more hairstyle selections by the user, thehairstyle database comprising hairstyles of celebrities sourced fromsocial media; displaying, on a display device, an image of the user'shead augmented with at least one set of hairstyles; synchronizing a userprofile based on the user with a publicly and privately availabledatabase of hairstyles; saving, by the user through the mobile computingdevice, hairstyles to the user profile; and accessing, by a hair stylistthrough a second mobile computing device, the user profile.
 13. Themethod of claim 12, further comprising: recommending, by the database,one or more hair products to the user based on the user profile andhairstyle selections by the user.
 14. The method of claim 13, furthercomprising selecting, by the user, one or more hair products, theselection being communicated in a graphical user interface of thedisplay device.
 15. The method of claim 12, wherein the method is storedin a set of instructions in a non-transitory computer readable storagemedium of the computing system.
 16. The method of claim 12, the step ofmoving the mobile computing device through one or more two- andthree-dimensional paths about the head further comprises: presenting aninstruction on the display device that causes the user to temporarilymaintain the mobile computing device at one location relative to apredetermined location of the head.
 17. The method of claim 16, furthercomprising temporarily positioning the camera of the mobile computingdevice towards the predetermined location to obtain the set of images.18. The method of claim 12, further comprising: moving the mobilecomputing device through one or more two- and three-dimensional pathsabout the head, and wherein the step of moving comprises positioning themobile device along a plurality of predetermined positions along apredetermined three-dimensional path.
 19. The method of claim 18,wherein the plurality of positions are determined automatically byanalyzing the set of images and identifying one or more landmarks of thehead of the user.
 20. The method of claim 12, further comprising:stitching together the set of images; and forming the user frameworkwith a three-dimensional map of the user's head, the three-dimensionalmap comprising the landmarks detected by analyzing the set of imagesthat are associated with the head.